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Abstract 


Let  {X  s  n£l,k2l}  be  an  array  of  row-wise  independent  random  elements 
nk 

in  a  Banach  space  of  type  p,  l<pS2.  The  convergence  of  i^a^X^  in 
probability  and  almost  surely  is  obtained  under  varying  moment  and  dis¬ 
tribution  conditions  on  {X^J*  In  particular,  laws  of  large  numbers  are 
obtained  for  triangular  arrays  of  random  elements.  Finally,  the  direct 
applications  of  these  results  in  obtaining  consistency  of  the  kernel  den¬ 
sity  estimates  are  indicated. 
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1.  Introduction  and  Preliminaries.  The  study  of  laws  of  large  numbers 

in  Banach  spaces  has  led  to  the  geometric  considerations  of  Banach  spaces 

such  as  Beck  convexity  (Beck  (1963)),  G  (Woycznski  (1973)),  and  type-p 

a 

(Hof fmann-J^rgensen  and  Pisier  (1976))-as  well  as  many  other  authors.  The 
study  of  probability  density  estimation  led  to  estimates  in  the  form  of 
averages  or  weighted  sums  of  random  variables  whose  values  are  in  function 
spaces  (Parzen  (1962)  and  Rosenblatt  (1971)).  As  a  application  of  the  law 
of  the  Iterated  logarithm  in  linear  measurable  spaces,  Kuelbs  (1978)  con¬ 
sidered  the  rates  of  convergence  in  these  density  estimates.  The  estimates 
are  not  always  averages  of  sequences  of  random  elements  in  a  Banach  space 
but  are  more  often  weighted  sums  of  arrays  of  random  elements  where  the 
weights  are  not  necessarily  Toeplitz  matrices. 

In  this  paper  the  convergence  of  weighted  sums  of  arrays  of  random 
elements  in  Banach  spaces  of  type  p  is  obtained  both  in  probability  and 
almost  surely.  As  corollaries  these  results  have  forms  of  Pruitt's  (1963) 
and  Rohatgi's  (1971)  results  for  Banach  spaces  and  also  have  extensions 
for  the  results  of  Padgett  and  Taylor  (1976).  However,  exact  statements 
of  the  theorems  are  related  to  the  possible  applications  for  density 
estimation.  T^aae  applications  are  indicated  in  Sections  3  and  4. 

Let  E  lenote  a  real  separable  Banach  space  with  norm  | |  | | .  Let 
(ft,A,P)  denoteVa  probability  space.  A  random  element  X  in  E  is  a 
function  from  fi  Unto  E  which  is  A-oeasurable  with  respect  to  the  Borel  subsets 
of  E.  The  expected  value  of  X  is  defined  to  be  the  Pettis  integral  (when 
it  exists)  and  is  denoted  by  EX.  The  moments  of  a  random  element  X  are 
E(||x||P)  where  E  id  the  expected  value  of  the  (real-valued)  random 


variable  ||x|| 


The  concepts  of  independence,  identical  distributions 


Definition  1:  A  separable  Banach  space  E  is  said  to  be  of  type  p 


for  every  finite  collection  of  independent  random  elements  X. 


E  with  mean  0  and  finite  pth  moments 


Hoffmann-J^rgensen  and  Pisier  (1976)  had  several  equivalent  statements 


for  type  p  spaces  including  the  strong  law  of  large  numbers.  For  the  re 


suits  of  this  paper,  the  following  analogue  of  Marclnkiewicz-Zygmund 


Inequality  by  Woycznski  (preprint)  will  be  used 


Proposition  1:  Let  l<p£2  and  q£l.  Then  E  is  of  type  p  if  and 


such  that 


X  with  0  means  and  finite  qth 


for  all  independent  random  elements  X. 


A  collection  of  random  elements  {X  )  in  E  is  said  to  be  stochastically 


bounded  by  a  random  variable  X 


If  the  random  elements  {X  }  are  identically  distributed,  then  they  are 


3 


1 


stochastically  bounded  by  the  random  variable  I  lx  II  for  any  a  . 

Oq  0 

Also,  by  Lemma  5.22  of  Taylor  (1978)  uniformly  bounded  rth  moments  (for 
some  r  >  0)  is  sufficient  for  stochastic  boundedness  by  a  random  variable 
X  and  for  the  existence  of  s(s  <r)  moments  for  the  random  variable  X. 

2.  Almost  Sure  Convergence  of  Weighted  Sums.  In  this  section  the 
almost  sure  convergence  for  Toeplitz  weighted  sums  of  arrays  of  random 
elements  is  obtained.  Recall  that  nil.kil}  is  a  Toeplitz  array 


(1)  ife  ank  "  0  for  each  k 

(ii)  I  anJc  I  -r  for  each  n 


(2.1) 

(2.2) 


where  it  can  be  assumed  that  T-l).  When  the  random  elements  {X  ,  }  In  E 

nk 

are  stochastically  bounded  by  the  random  variable  X  and  E|x|  <®,  then 

^-lElla»k!!okllSJk.lla„klElXlSElXl 

and  J^a^X^  is  convergent  almost  surely  for  each  n.  Theorem  1  will  be 
stated  for  complete  convergence  (which  implies  almost  sure  convergence) 
since  the  proof  will  consist  of  showing  Y“  ,  PL  |  1 7°°  ,a  ,X  ,  ||  u]<«  for 
each  e  >  0. 


Theorem  1:  Let  (X^)  be  an  array  of  random  elements  in  a  Banach 

space  of  type  p,  l<ps2,  which  are  row-wise  Independent  and  such  that 

EX^  “  0  for  all  k  and  n.  If  {X^}  are  stochastically  bounded  by  a  random 

variable  X  and  if  (a^}  is  a  Toeplitz  array  such  that 

max | a  |  »0(n“Y),  y  >  0,  then  E|x|1+1^Y  <  »  implies  that 
k  nK 


<  ® 


. 1 


4 


M I"  /nkXnkH  0  completely. 

k“l 

The  proof  of  Theorem  1  Is  accomplished  In  three  lemmas.  The  first 
two  lemmas  follow  directly  from  Pruitt  (1966)  and  Rohatgi  (1971)  since 

I lankXnkl l“lankl I lXnkl I  and  { lankl }&{ 1 1 Xnkl I }  are  weights  and  random 
variables  satisfying  the  corresponding  hypotheses. 


Lemma  2;  If  E|X|^+^y<*»  and  max|ankJ  SBn””* ,  then  for  every  e  >  0 

k 

I^«ip£ I lankXnkl I  2  €  for  some  k]  <  "• 

Lemma  3:  If  e|x|^+^Y<®  and  maxla^jj  sBn”^,  then  for  a  <  y/2(y  +  1), 

k 

I  a^X^I  |  J:  n-0  for  at  least  two  values  of  k]<®. 

Lemma  A:  If  EXQk  *  0  for  all  n  and  k,  E|x|1+1^Y<  »,  and 
max |  a  |  £Bn~Y,  then  for  every  e  >  0, 


Ci«i 


l^k®lankXnk^[| 


ankXnk* 


n~BJ 


U  e]  <  « 


where  0  <  o  <  y. 


.-1 


Proof :  When  ank  ■  0,  then  |ank|  is  understood  to  be  ®.  Since 


EX 


nk 


lE(Wc||.nkXnkll«n-3)| 


a<l,^*,|lt||*JI*0«Wtl'13> 


0, 


IxIan^B'V 


| x | dP  C  j  X  |  <  x ldx • 


(2.3) 


Thus,  y  >  a  yields  ||E(X  Ir 


nk  tNankXnkN<n  3 


)  |  |  -*•  0  uniformly  in  k  as  n 


Hence , 


1  ^k-lSl^Vt  |  [»nkXnkl  I  <  n"0]’  1 1 

‘^i!*J£(N^,kllIr||.iikx  ||  <»•«]>  *  0  <2-‘> 


as  n -*■•>.  Define, 


Znk  “  XnkX[||a.X  .  ||  <n^rE(XnkI[||a„1,X„J|  <n^])*  (2.5) 


nk  nk 


nk  nk' 


Note  that  EZnk  -  0,  E|  | Znk|  | 1+1/y  s  K^l  | Xnk|  |  1+1/ys  K1E|x|1+1/y-  d  for 
some  d >  0,  and  ||a  Z  ,  ||  £  2n~a .  From  (2.4) 


nk  nk 


C 1  ^k-lWt  I  |ankXnk|  |  <  n^]1  >  >  c] 


C^^k-lankZnkl  I  2  2 3 


for  sufficiently  large  n.  For  each  n,  pick  s(n)  so  that 


l  ^  -2 

a  hn  . 


(2.6) 


-k*s(n)+l'  nk 


Let  v  be  chosen  so  that 


s  •  —  is  an  integer  and  v>  [  (1-  — )  (2y)  ]”X. 
P  P 

_  f  II.-  *•  / 1  I  V®(®) .  »  ll  2v\  j  _  ...j 


(2.7) 


Since  ||ankZnk||  zlvT01,  E(||£a_”  a„vz..  J  I  V)  is  finite.  Using  Proposition  1 


k*l  nk  nk 


E<H£lankZnkM  V>SC2E  <£l  HankZnkNP)P3 
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-C2„  I 


(2.8) 


kl . ks  J*1 


j  j 


In  the  general  term  of  (2.8),  let 


q1  of  the  k's  =  ...  .q^  of  the  k's  * 

of  the  k's  -  b^  ...  ,rt  of  the  k's  -  bt 

where  p  i  pqi  S  1  +  ^,  pr ^  >  1  +  ^,  and 


Thus,  In  (2.8) 


K  S  II...  Z„.  I|PJ  ■  E[j1ll*nc1ZnClll',,1j;ill*nb^nbjl 


.  Pr4 


j-1  Dkj 


m  P9.,  P1!.,  t  P 

^<llzncjl  )]:iI1E(llanbjZ.bjll  J)] 

*  <l«1E|x|1+1/l'>”ft(  i?  |.  I  |anc  |Ml  S 


1-1 

’,<Ji|,»»J|1+I,'K2n"')'ri 


S(l+d)t'hn(  n  \anr  I)  (  n  |anK  |)(Bn-Y) 
1=1  ncl 


Prj-1-  T 

«(2n^)Z3-l<Prj'1'  7*. 


j-1  nbj 


(2.9) 


For  the  last  expression  in  (2.9),  the  power  of  n  ^  is 


Y^l-l(pqi‘1)  +  1  +  ^J-l^j"1"^ 


(2.10) 


If  t  2 1,  then 


Y^i-l(pqi_1)  +  t  + 


7 


2  1  +  a^_1(Prj-l-l/Y)^l  + 


a6 


(2.11) 


for  some  fixed  6 
then 


>  0  since  is  an  integer  greater  than  ^  If  t  -  0, 


yXJ-1  CMi-iJ  +  *  +  a^-l(prj-1’  Y> 

-  yI^Cpv1) 

“Y(2v-f) 

-  y(2v) (1“ p  >  >  1 

by  (2.7).  By  (2.7)  and  (2.11)  r  -  min{y(2v)  (1-^) ,  l  +  a6)  >  1.  From  (2.8) 
and  (2.9) 

«ll7£H\Akll*,>«V‘r 

where  the  constant  Kj  depends  only  on  d,Y*P,  and  B. 

Next, 

In=lp£  I  Hk«lankZnlJ  ^  *  2] 

+  ^n-lPi;l  ^k-s(n)+lankZnk'  '  *  43 

*  (r>2vi:.1«liI^>+Vnkiiz',i 

4  r>oo  r*a> 


+  c  ^n-l^k-s(n)+lE* lankZnk' 


2\> 


5  (e)  ^n-lK2n  +  E^n-l^k-s(n)+llankl (1+d) 


8 


A  version  of  Rohatgi’s  (1968)  result  for  type  p,  1  <ps2,  spaces  is 
obtained  directly  from  Theorem  1  as  a  corollary.  It  is  interesting  to 
observe  that  the  moment  condition  is  related  to  the  weights  rather  than 
the  type  p  condition.  Typically,  y  s  1»  and  hence  1  +  l/-y«:2ip. 


Corollary  1:  Let  (Xn>  be  a  sequence  of  independent  random  elements 
in  a  Banach  space  of  type  p,  1  <ps2,  such  that  EX^  ■  0  for  each  n,  and 
let  (Xn>  be  stochastically  bounded  by  a  random  variable  X.  If  (a^)  is 
a  Toeplitz  array  such  that  max|an^|»0(n  Y),  y  >  0,  then  E|x|1+1^<® 
Implies  that 


I  ^lank*J  I  *  0  comPletely- 


9 


The  geometric  property  of  E  significantly  relaxes  the  condition  im¬ 
posed  in  Padgett  and  Taylor  (1976),  Theorem  2,  to  extend  Rohatgi's  results 
to  Banach  spaces.  The  type  p  condition  is  not  needed  in  the  corresponding 
results  for  independent,  identically  distributed  random  elements  since 
the  uniform  truncation  is  accomplished  by  the  identical  distributions. 

3.  Convergence  in  Probability  and  Density  Estimates.  Convergence 
•in  probability  of  the  weighted  sums  will  be  obtained  in  this  section 
under  relaxed  conditions  on  the  summability  of  the  weights.  The  general 
application  of  these  results  to  density  estimates  will  be  indicated. 

Theorem  2;  Let  {X^}  be  an  array  of  row-wise  independent  random 

elements  in  a  Banach  space  E  of  type  p,  1  <ps2.  Let  EX^-O  for  each  n 

and  k  and  let  {X^}  be  stochastically  bounded  by  a  random  variable  X. 

Let  {a  }  be  an  array  of  constants  such  that  max|a  ,  |-*-0  as  n-*-«°  and 
nk  lsk<n  nKL 

^k-l^nkl^1,  £or  a11  n 
where  lsr<p.  If  E(|x|r)<®,  then 


l*^k«lankXnJ  ^  0  111  Probabillty 


The  proof  consists  of  incorporating  the  techniques  of  Theorem  1  into 
the  proof  of  Theorem  5.3.2  of  Taylor  (1978),  and  will  be  omitted. 

In  the  general  density  estimation  problem  X^,...,Xn  are  independent 
random  variables  with  the  same  density  function  f.  The  kernel  estimate 
for  f  is  given  by 


fn<‘> 


(3.1) 
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where  K  Is  an  arbitrarily  chosen  density  and  h^+0  as  There  are 

numerous  choices  for  K  and  h  .  Here,  we  will  assume  that  K(t)  is  a 

n 

bounded  (integrable)  kernel  with  compact  support.  Let  w(t)  be  a  bounded 
weight  function  such  that 


/| f (t) |Pw(t)dt  <“  (3.2) 

log  h 

for  some  -  <  lim  inf  Cl+-= - -J.  Typically,  h  -  n  . 

p  n  log  n  n 

Hence,  p  >  where  0  <  6  <  %.  Denote 

E  -  { g :  g:R+R  and  f°\ g(t) |Pw(t)dt < »} .  (3.3) 

•CO 

Then,  E  is  a  separable  Banach  space  of  type  min{2,p}  with  norm 

n  1/p 

I  lei  |“</|g<t)I PwC«:)dt>  •  (3-4) 


Since  K  is  bounded  with  compact  support,  {K(-r — ),....K(~T — H  are 


i.l.d.  random  elements  in  E  for  each  n.  For  q£l 


t-X,  <1  ^  t-X  P  o/n 

E(||K(-iri)||  )-EC(/^|K(-iri>  |  w(t)dt)q/P] 
n  n 

-  E[ (/  |K(s)|Pw(X1+shn)hnds)q/p] 


s  (bdd  K)q[(bdd  w)(b-a)hn3 


q/p 


<  °». 


(3.5) 


t-X, 


1 

From  (3.5)  the  expected  value  E[K(~ jj — )]eE,  and  the  random  elements 
t-X.  t-X  n 

{K(  -)-E[K(  .  -■)]:  k  -  l,2,...,n  and  n 2 1}  are  stochastically  bounded 

n  n 

by  a  random  variable  X.  Let  *nk  “  ^  for  1  s  n  and  ank  “  0  for  k  >  n. 
Choose  r  so  that  Is  r  <p  and 
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Cl1**1 


r  r  1  if  1-r .  -r 

■nl— r— J  ■  n  h  £  r 
nh  n 


(3.6) 


for  all  n.  Under  the  conditions  of  (3.1)  to  (3.6),  Theorem  2  yields 


I  bir  Iw_.  -  E«(-rl»ll*o 


t- 


Xl. 


•nh  ^k-1 

n  n  n 

In  probability.  The  troublesome  aspect  is  showing  that 

|  |^pE(K(— jj-^))-f(t)|  !-*•  0. 
n  n 


(3.7) 


(3.8) 


T-omma  s ?  Let  f  be  piecewise  continuous,  bounded  on  each  compact  sub¬ 


set,  and  lim  f(s)=0.  Then 
isk® 


t-X 

|j^E(K(-lr^))-f(t)||  -0. 
ti  n 


Proof :  First, 


t-X 


|  |ji-E(K(-^-^))-f(t)  |  |P 


r  lir  A(x£)f(8>d8“f(t)lPw(t)dt 


'h  >  "'h 
n  -®  n 


-  rif  K(y)[f (t-yh  )-f (t)]dy| Pv(t)dt. 
_ a  n 


(3.9) 


The  integral  with  respect  to  t  in  (3.9)  may  need  to  be  broken  into  several 
(improper)  integrals,  so  that  (w.l.o.g.),  it  can  be  assumed  that  f  is  con¬ 
tinuous  at  each  t.  Thus,  for  each  y  and  each  t,  f (t-yhn>+f (t)  as  n^®. 
Choose  N1  so  that  lyhj  si  for  all  n^Nj.  Hence, 

| f (t-yh  )-f (t) |  £  |f(t)|  +  sup  |f(t-s)|  ST(t) 
n  -lsssl 


for  each  t. 


Hence, 
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b 

/  K(y) [f (t-yh  )  -  f(t)]dy  -*•  0  (3.10) 

a 

pointwise  In  t.  Choose  a  and  B  so  that 

a+1 

/  £(t)dt  £ e/bdd  w. 


OD 

/  f(t)dt £ e/bdd  w,  and 
B-l 

f(t)  s  j  for  trf[a+l,B-l3.  (3.11) 


Then  for  n  2: 

00  l) 

/  I  f  K(y)[f(t-yh  )-f(t)]dy|Pw(t)dt 

_oo  a  n 

a  b 

£  /  |  /K(y)[f (t-yhn)-f(t)]dy|w(t)dt 
oo  a 

B  b 

+  /  |  /  K(y)[f(t-yhn)-f (t)]dy|Pw(t)dt 
a  a 
00  l) 

+  /  I  / K(y) [ f (t-yh  )-f (t) ] | w(t)dt .  (3.12) 

B  a 

The  first  term  of  (3.12)  becomes 
o  b 

/  I  /K(y)[f(t-yh  )-f(t)]dy|w(t)dt 

-oo  a 

o  b  a  b 

S  bdd  w (  /  /  K(y)f(t-yhn)dydt  +  /  /  K(y)f (t)dydt) 

-oo  a  a 

<  c  +  e  (3.13) 


by  (3.11)  and  an  Interchange  of  Integrals.  Similarly,  the  third  term  of 
(3.12)  Is  less  than  2c.  The  second  term  of  (3.12)  converges  to  zero  by 
the  dominated  convergence  theorem.  Hence,  there  exists  N  such  that 
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i  fc-xi 

||jp  E(K(— (t)  |  | P  <  5e 
n  n 

for  all  n  i  N.  Ill 

Theorem  2  and  Lemma  5  provides  the  consistency 

OD 

|  |  £  (t)-f(t)||-J  If  (t)-f(t)|Pv(t)dt  -  0 

In  probability  where  the  pth  integrated  difference  with  respect  to  an 
arbitrary  might  function  w(t)  is  related  to  the  order  of  h^.  Before 
proceeding  to  the  complete  convergence  of  the  estimates,  it  is  important 
to  note  that  w(t) £  1  is  often  assumed  and  that  various  combinations  of 
conditions  on  K  and  f  will  yield  Lemma  5. 

A.  A  Strong  Law  of  Large  Numbers  Application.  In  this  section  a 
strong  law  of  large  numbers  is  obtained  for  arrays  of  random  elements 
in  type  p  spaces.  This  result  substantially  improves  the  density 
estimation  application  in  the  previous  section.  The  proof  of  Theorem  3 
will  make  use  of  the  following  lemma  (Corollary  2.1  of  Woycznskl  (pre¬ 
print)). 

Lemma  6:  Let  E  be  of  type  p,  1  <  p  %  2,  and  let  qil.  If  (Xn)  are 
independent,  identically  distributed  random  elements  in  E  with 
E(|  |X_  1 1^)  <  *  end  EX,  -  0,  then 

«l  IZS.AI  I")  *  c,.«e(|  11,11",  <*•» 

for  some  constant  C2  independent  of  n. 

Only  the  major  points  of  the  proof  of  Theorem  3  will  be  indicated. 
Again,  the  proof  will  use  higher  moment  conditions  on  the  random  elements 
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than  the  type  of  the  space. 

Theorem  3;  Let  {Xnk}  be  an  array  of  row-vise  independent.  Identically 
distributed  random  elements  in  a  Banach  space  of  type  p,  l<ps2,  and  let 
EXnk«0  for  each  n  and  each  k.  If  for  some  q  i.  1 

(4.2) 

then 


I  l^k-lXnkl  I  -*■  0  completely. 

Proof:  For  each  e  >  0  and  each  n 

"I  l&Akl  I  »  ‘1  S  n-^^E<|  ir^Xnk|  |M, 

4  n->>’£-,>’c2n’E<||Xnl||M) 

by  Lemma  6.  Thus, 

^n-lPCHn^k-lXnk^  *  e] 

*  e-pqC2i;.1E(||Xnl||P‘»)/n^P-1)<- 

by  (4.2). 

In  the  density  estimation  problem,  let 


(4.3) 


/// 
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Leona  5  provides  for  the  convergence 

n  n 

In  verifying  the  moment  condition  (4.2),  Inequality  (3.5)  will  be  used. 
For  any  q  2  1 


E(||Xnll|Pq)iE[(||^(^i)||  +  ||^E(K(-ir^)||)Pq) 

n  n  j  n  n 

S  hn"Pq[E(||K(i^)||pq)pq  +  ||e(K(1A))||]P<1 

n  _l  l  n 

s  h  "pq[(c.h  q)pq  +  (c.h  p)]pq 

n  h  n  on 

S  h  ”PqCAh  q  -  C.h  q(1_p)  (4.4) 

n  6  n  6  n 


where  C^,  C^,  and  Cg  are  constants  depending  on  the  bounds  of  K  and  w  and 
the  support  of  K,  [a,b],  and  the  values  of  p  and  q.  Hence, 

E(|  |xnl|  |PW<',-1)  s  c6h;(1-p>/^<p-» 

.  C.(nh  )'',<P"1).  (4.5) 

o  n 

Thus,  if  hQ  is  of  the  order  n  5  for  any  0<  6  <  1  there  exists  a  q  2 1  such 
that  the  terms  in  (4.5)  sum.  By  Theorem  3  and  Lemma  5 

/|fn(t)-f(t)|Pw(t)dt  -*■  0  completely 

for  any  p,  l<pS2,  unrelated  to  the  hn's  when 


T"  ,(nh  > 

'n-1  n 


<  • 
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for  some  a >  0.  Not  only  is  complete  convergence  obtained,  but  the  h  ' s 

n 

are  much  more  general.  This  Improvement  is  mainly  due  to  the  probability 
cancellation  in  type  p  spaces,  and  the  pth  Integrated  norm  (in  (A. 4)). 

Inequality  (A. 3)  suggests  an  immediate  weak  law  of  large  numbers  for 
arrays  of  random  elements  in  type  p  spaces. 

Theorem  A:  Let  {X^}  be  an  array  of  row-wise  Independent,  identically 

distributed  random  elements  in  a  Banach  space  of  type  p,  l<ps2,  and  let 

EX  ,  *0  for  each  n  and  each  k.  If  for  some  q2l 
nk 

n-q(p-l)E(||x^||pq)  _  (4.6) 

then 

I ln£k-lXnJ I ^  0  in  Probability. 

The  condition  of  row-wise  lndentically  distributed  random  elements  in 
Theorems  3  and  A  is  not  necessary  since  Proposition  1  could  be  used  instead 
of  Leona  6.  However,  expressions  (A. 2)  and  (A. 6)  would  appear  much  more 
complicated. 

5.  Acknowledgements.  Sincere  thanks  go  to  Wojbor  Woycznskl  for 


discussions  on  the  Marclnklewicz-Zygmund  inequality  in  type  p  spaces  and 
for  suggesting  the  framework  for  those  results. 
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